This project aims to build a Deep Convolutional Generative Adversarial Network (DCGAN) that trains on images of Monet paintings to capture their qualities and then generate Monet-esque images. Several techniques will be showcased such as image augmentation for training diversification, an iterative approach to DCGAN architecture resulting in multiple model runs and submissions, and a discussion on common issues that can arise during GAN training.
This project derives from the Kaggle competition with the name Gan Getting Started (I'm Something of a Painter Myself), found here: https://www.kaggle.com/competitions/gan-getting-started/overview
You can find this project at the github repo: https://github.com/chill0121/Kaggle_Projects/tree/main/Adversarial_Painting
Color images (256 x 256 pixels) extracted from histopathologic scans of lymph node sections. These 96 x 96 images are patches of a whole slide image.
Data Info:
We recognize the works of artists through their unique style, such as color choices or brush strokes. The “je ne sais quoi” of artists like Claude Monet can now be imitated with algorithms thanks to generative adversarial networks (GANs). In this getting started competition, you will bring that style to your photos or recreate the style from scratch!
Computer vision has advanced tremendously in recent years and GANs are now capable of mimicking objects in a very convincing way. But creating museum-worthy masterpieces is thought of to be, well, more art than science. So can (data) science, in the form of GANs, trick classifiers into believing you’ve created a true Monet? That’s the challenge you’ll take on!
The Challenge: A GAN consists of at least two neural networks: a generator model and a discriminator model. The generator is a neural network that creates the images. For our competition, you should generate images in the style of Monet. This generator is trained using a discriminator.
The two models will work against each other, with the generator trying to trick the discriminator, and the discriminator trying to accurately classify the real vs. generated images.
Your task is to build a GAN that generates 7,000 to 10,000 Monet-style images.
Note: Monet-style art can be created from scratch using other GAN architectures like DCGAN. The submitted image files do not necessarily have to be transformed photos.
MiFID Submissions are evaluated on MiFID (Memorization-informed Fréchet Inception Distance), which is a modification from Fréchet Inception Distance (FID).
The smaller MiFID is, the better your generated images are.
What is FID? Originally published here (github), FID, along with Inception Score (IS), are both commonly used in recent publications as the standard for evaluation methods of GANs.
Amy Jang, Ana Sofia Uzsoy, Phil Culliton. (2020). I’m Something of a Painter Myself. Kaggle. https://kaggle.com/competitions/gan-getting-started
import os
import sys
import zipfile
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from PIL import Image
import tensorflow as tf
import torch
print(f"Python version: {sys.version}")
packages = [pd, np, sns, tf, torch]
for package in packages:
print(f"{str(package).partition('from')[0]} using version: {package.__version__}")
Python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] <module 'pandas' using version: 2.2.2 <module 'numpy' using version: 1.26.4 <module 'seaborn' using version: 0.12.2 <module 'tensorflow' using version: 2.16.1 <module 'torch' using version: 2.4.0
# Set directories
current_wdir = os.getcwd()
data_folder = '/kaggle/input/gan-getting-started/'
img_size = 256
batch_size = 16
rand_seed = 11
monet = tf.keras.utils.image_dataset_from_directory(f"{data_folder}/monet_jpg",
label_mode = None,
image_size = (img_size, img_size),
batch_size = batch_size)
monet = monet.map(lambda x: (x / 127.5) - 1)
Found 300 files.
As expected we're showing 300 images of Monet paintings.
To get an idea what the paintings looks like, we should plot a few here.
for img in monet:
fig, ax = plt.subplots(2, 3, sharex = True, sharey = True)
img = ((img.numpy() + 1) * 127.5).astype(np.uint8)
for i in range(6):
ax[i // 3, i % 3].imshow(img[i])
break
Training a deep learning model off of 300 images isn't that many.
Let's add some image augmentation/transformations to the dataloader, effectively increasing the size of the dataset size.
Transformations:
# Transformation object.
data_augmentation = tf.keras.Sequential([
tf.keras.layers.RandomFlip("horizontal_and_vertical"),
tf.keras.layers.RandomRotation(0.1),
tf.keras.layers.RandomCrop(img_size, img_size)])
# Autotune to set buffer size.
AUTOTUNE = tf.data.AUTOTUNE
# Function to setup the transformations within the dataloader.
def prepare(ds, shuffle = False, augment = False):
if shuffle:
ds = ds.shuffle(1000)
if augment:
ds = ds.map(lambda x: (data_augmentation(x, training=True)),
num_parallel_calls = AUTOTUNE)
# Prefetch buffer necessary here to ensure proper loading during training.
return ds.prefetch(buffer_size = AUTOTUNE)
Now we can prepare the dataloader with the function we just created.
monet_transformed = prepare(monet, shuffle = True, augment = True)
Let's plot a few images and see the transformations in practice.
for img in monet_transformed:
fig, ax = plt.subplots(2, 3, sharex = True, sharey = True)
img = ((img.numpy() + 1) * 127.5).astype(np.uint8)
for i in range(6):
ax[i // 3, i % 3].imshow(img[i])
break
Looks good, we can obviously see the rotations and a few flips even without seeing the originals.
Let's load the images outside the dataloader so we can avoid the batching and do a little EDA. Since there are only 300, this isn't too much of a memory hit.
monet_files = os.listdir(f"{data_folder}/monet_jpg")
monet_array = np.zeros((len(monet_files), 256, 256, 3)) # 4D Array with shape (n_images, height, width, channels)
for i, file in enumerate(monet_files):
img = Image.open(f"{data_folder}/monet_jpg/{file}")
monet_array[i] = np.asarray(img)
Since a Generative Adversarial Network is going to be trained to generate images in Monet's style from scratch, it might be good to inform ourselves of what color intensities are present within this dataset.
# Find the mean pixel intensity of each image (down to one value along all axes).
mean_intensity = monet_array.mean(axis = 1).mean(axis = 1).mean(axis = 1)
sns.histplot(mean_intensity, binwidth = 5, kde = True)
plt.title('Average Pixel Intensity')
plt.show()
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
Look at the mean pixel intensity value of each image here, we can see that there aren't many images skewing white (close to 255, right) or black (close to 0, left). Most images sit near the middle around 130 (127.5 would be center).
mean_intensity_rgb = monet_array.mean(axis = 1).mean(axis = 1)
# Plot the distribution of the mean intensities of individual RGB channels.
rgb_dict = {0:'Red', 1:'Green', 2:'Blue'}
for i in rgb_dict.keys():
sns.histplot(mean_intensity_rgb[:,i], color = rgb_dict[i], binwidth = 5, kde = True)#, ax = ax[i])
plt.title('Average Pixel Intensity of Each Channel (RGB)')
plt.legend(list(rgb_dict.values()))
plt.show()
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
It seems Monet painted with a wide variety of colors and this should be present in the final generations.
Instead of performing style-transfer like pix2pix or with a CycleGAN, I plan to train a DCGAN to generate Monet-like images and see how well it performs in this competition.
To review, a DCGAN is a variation of the more general GAN models that utilizes deep convolutional layers in both the discriminator and generator. This, of course, means this type of model is adept at image data.
DCGANs are comprised of two models, the Discriminator whose purpose is to identify (discriminate) real images from fake ones, and the Generator whose role is to generate images that hopefully imitate real images well enough to fool the Discriminator. The two goals of these models are diametrically opposed to each other, meaning if one is succeeding, the other is failing. This can be seen in the loss of each model, but there is no perfect metric to know when the model has converged, overfit, or suffered mode collapse (more on this later). This fact makes tuning a GAN model fairly subjective and requires balancing the two models so that they converge into a working model which be a difficult and time-consuming process.
latent_dim = 512
epochs = 250
num_img_saved = 1
# Image dir saved per epoch.
! mkdir /kaggle/working/Generated_Images
# Image dir saved for submission.
! mkdir /kaggle/working/images
/opt/conda/lib/python3.10/pty.py:89: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock. pid, fd = os.forkpty()
Now we can initialize the Discriminator (sometimes called the Critic depending on application).
Generally the role of the discriminator isn't unlike a CNN used in classification: downsample an image into a feature space and classify it.
It's very important to not only plan the architectures of the discriminator and generator (input shapes vs output shapes, filter size, number of filters, strides, etc), but also choose the proper activation functions, especially for the last layers.
Since we're using Binary Cross Entropy as the loss function, the last layer can either use a sigmoid activation function or ensure the loss function specifies from_logits = True.
discriminator = tf.keras.Sequential(
[tf.keras.Input(shape = (img_size, img_size, 3)),
tf.keras.layers.Conv2D(64, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
#tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(128, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(128, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(1)],
name = 'Discriminator')
discriminator.summary()
Model: "Discriminator"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ conv2d (Conv2D) │ (None, 128, 128, 64) │ 4,800 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu (LeakyReLU) │ (None, 128, 128, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_1 (Conv2D) │ (None, 64, 64, 128) │ 204,800 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization │ (None, 64, 64, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_1 (LeakyReLU) │ (None, 64, 64, 128) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_2 (Conv2D) │ (None, 32, 32, 128) │ 409,600 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_1 │ (None, 32, 32, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_2 (LeakyReLU) │ (None, 32, 32, 128) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_3 (Conv2D) │ (None, 16, 16, 256) │ 819,200 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_2 │ (None, 16, 16, 256) │ 1,024 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_3 (LeakyReLU) │ (None, 16, 16, 256) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_4 (Conv2D) │ (None, 8, 8, 256) │ 1,638,400 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_3 │ (None, 8, 8, 256) │ 1,024 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_4 (LeakyReLU) │ (None, 8, 8, 256) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ flatten (Flatten) │ (None, 16384) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 16384) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (Dense) │ (None, 1) │ 16,385 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 3,096,257 (11.81 MB)
Trainable params: 3,094,721 (11.81 MB)
Non-trainable params: 1,536 (6.00 KB)
Now for the Generator.
This model requires us to architect a random latent vector as an input, then upsample this vector through an emergent feature space to the size of a real image which will then be input into the Discriminator during training. Several different model architectures can work here and many variations of GANs can be identified by the Generator's shape.
Here, we have a choice of activation function in the last layer which mostly depends on the range of pixel values of our images. In this case we normalized the images into the space of [-1, 1] so the hyperbolic tangent (tanh) activation function has been chosen. If we had normalized from [0, 1], the sigmoid function would be appropriate. Tanh seems to be accepted practice by performing the best in GANs, however.
# # Shallow DCGAN
# generator = tf.keras.Sequential(
# [tf.keras.Input(shape = (latent_dim, )),
# tf.keras.layers.Dense(16 * 16 * latent_dim),
# tf.keras.layers.Reshape((16, 16, latent_dim)),
# tf.keras.layers.Conv2DTranspose(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# #tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(512, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(512, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2D(3, kernel_size = 7, padding = 'same', activation = 'tanh')],
# name = 'Generator')
# generator.summary()
# # Flipped DCGAN
# generator = tf.keras.Sequential(
# [tf.keras.Input(shape = (latent_dim, )),
# tf.keras.layers.Dense(4 * 4 * latent_dim),
# tf.keras.layers.Reshape((4, 4, latent_dim)),
# tf.keras.layers.Conv2DTranspose(512, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# #tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(128, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2DTranspose(128, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
# tf.keras.layers.BatchNormalization(),
# tf.keras.layers.LeakyReLU(negative_slope = 0.2),
# tf.keras.layers.Conv2D(3, kernel_size = 7, padding = 'same', activation = 'tanh')],
# name = 'Generator')
# generator.summary()
# Deeper DCGAN
generator = tf.keras.Sequential(
[tf.keras.Input(shape = (latent_dim, )),
tf.keras.layers.Dense(16 * 16 * latent_dim),
tf.keras.layers.Reshape((16, 16, latent_dim)),
tf.keras.layers.Conv2DTranspose(512, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
#tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2DTranspose(256, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2DTranspose(128, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2DTranspose(64, kernel_size = 5, strides = 2, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(32, kernel_size = 5, padding = 'same', use_bias = False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(negative_slope = 0.2),
tf.keras.layers.Conv2D(3, kernel_size = 7, padding = 'same', activation = 'tanh')],
name = 'Generator')
generator.summary()
Model: "Generator"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ dense_1 (Dense) │ (None, 131072) │ 67,239,936 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ reshape (Reshape) │ (None, 16, 16, 512) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_transpose │ (None, 32, 32, 512) │ 6,553,600 │ │ (Conv2DTranspose) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_5 (LeakyReLU) │ (None, 32, 32, 512) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_transpose_1 │ (None, 64, 64, 256) │ 3,276,800 │ │ (Conv2DTranspose) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_4 │ (None, 64, 64, 256) │ 1,024 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_6 (LeakyReLU) │ (None, 64, 64, 256) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_transpose_2 │ (None, 128, 128, 128) │ 819,200 │ │ (Conv2DTranspose) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_5 │ (None, 128, 128, 128) │ 512 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_7 (LeakyReLU) │ (None, 128, 128, 128) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_transpose_3 │ (None, 256, 256, 64) │ 204,800 │ │ (Conv2DTranspose) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_6 │ (None, 256, 256, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_8 (LeakyReLU) │ (None, 256, 256, 64) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_5 (Conv2D) │ (None, 256, 256, 32) │ 51,200 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ batch_normalization_7 │ (None, 256, 256, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ leaky_re_lu_9 (LeakyReLU) │ (None, 256, 256, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_6 (Conv2D) │ (None, 256, 256, 3) │ 4,707 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 78,152,163 (298.13 MB)
Trainable params: 78,151,203 (298.12 MB)
Non-trainable params: 960 (3.75 KB)
Now we need a way for these two models to be set up opposing each other, in an adversarial way. The following class will handle the training program.
# GAN Class was altered from https://keras.io/examples/generative/dcgan_overriding_train_step/
class GAN(tf.keras.Model):
def __init__(self, discriminator, generator, latent_dim):
super().__init__()
self.discriminator = discriminator
self.generator = generator
self.latent_dim = latent_dim
# self.seed_generator = tf.keras.random.SeedGenerator(rand_seed)
def compile(self, d_optimizer, g_optimizer, loss_fn):
super().compile()
self.d_optimizer = d_optimizer
self.g_optimizer = g_optimizer
self.loss_fn = loss_fn
self.d_loss_metric = tf.keras.metrics.Mean(name = "D_Loss")
self.g_loss_metric = tf.keras.metrics.Mean(name = "G_Loss")
@property
def metrics(self):
return [self.d_loss_metric, self.g_loss_metric]
def train_step(self, real_images):
# Sample random points in the latent space.
batch_size = tf.keras.ops.shape(real_images)[0]
random_latent_vectors = tf.keras.random.normal(
shape=(batch_size, self.latent_dim))#, seed=self.seed_generator)
# Decode the latent space to a fake image.
generated_images = self.generator(random_latent_vectors)
# Zip with real images.
combined_images = tf.keras.ops.concatenate([generated_images, real_images], axis=0)
# # Assemble labels discriminating real from fake images.
# labels = tf.keras.ops.concatenate([tf.keras.ops.ones((batch_size, 1)), tf.keras.ops.zeros((batch_size, 1))], axis=0)
# # Add noise to labels.
# labels += 0.05 * tf.random.uniform(tf.shape(labels))
# Create labels for real (1) and fake (0) images
real_labels = tf.ones((batch_size, 1))
fake_labels = tf.zeros((batch_size, 1))
# Apply label smoothing: change real labels from 1 to a value like 0.9
smoothed_real_labels = real_labels * 0.9 # Real label smoothing
combined_labels = tf.concat([smoothed_real_labels, fake_labels], axis=0)
# Add some noise to the labels (optional but can help)
combined_labels += 0.05 * tf.random.uniform(tf.shape(combined_labels))
# Train the discriminator.
with tf.GradientTape() as tape:
predictions = self.discriminator(combined_images)
d_loss = self.loss_fn(combined_labels, predictions)
grads = tape.gradient(d_loss, self.discriminator.trainable_weights)
self.d_optimizer.apply_gradients(
zip(grads, self.discriminator.trainable_weights))
# Sample random points in the latent space.
random_latent_vectors = tf.keras.random.normal(
shape=(batch_size, self.latent_dim))#, seed=self.seed_generator)
# Assemble misleading labels that show all real images.
misleading_labels = tf.keras.ops.zeros((batch_size, 1))
# Train the generator.
with tf.GradientTape() as tape:
predictions = self.discriminator(self.generator(random_latent_vectors))
g_loss = self.loss_fn(misleading_labels, predictions)
grads = tape.gradient(g_loss, self.generator.trainable_weights)
self.g_optimizer.apply_gradients(zip(grads, self.generator.trainable_weights))
# Update loss metrics.
self.d_loss_metric.update_state(d_loss)
self.g_loss_metric.update_state(g_loss)
return {"d_loss": self.d_loss_metric.result(),
"g_loss": self.g_loss_metric.result()}
Here we can create a callback for tensorflow to use at the end of each epoch. This class and function will save randomly generated images after each epoch -- This is incredibly important for the subjective tuning and analysis of how well the model performs as mode collapse or non-convergence can happen at any time during the training of a GAN.
class GANMonitor(tf.keras.callbacks.Callback):
def __init__(self, num_img=3, latent_dim=latent_dim):
self.num_img = num_img
self.latent_dim = latent_dim
# self.seed_generator = tf.keras.random.SeedGenerator(rand_seed)
# Save a few generated images at each epoch.
def on_epoch_end(self, epoch, logs=None):
random_latent_vectors = tf.keras.random.normal(shape=(self.num_img, self.latent_dim))
generated_images = self.model.generator(random_latent_vectors, training = False)
generated_images = (generated_images + 1) * 127.5
for i in range(self.num_img):
img = tf.keras.utils.array_to_img(generated_images[i])
img.save(f"{current_wdir}/Generated_Images/{epoch+1:03}_{i}.jpg")
# In the case of mode collapse for kaggle submissions.
if epoch > (epochs - 21):
# Sorry for anyone who reads this next section, it's convoluted in its methods.
n = 400
random_latent_vectors = tf.keras.random.normal(shape=(n, 512))
for i in range(int(n/100)):
# print('####', i*100, (i+1)*100)
generated_images = self.model.generator(random_latent_vectors[i*100: (i+1)*100], training = False)
generated_images = (generated_images + 1) * 127.5
n_range = np.linspace(i*100, ((i+1)*100)-1, int(n/4), dtype = int)
for i, j in enumerate(n_range):
# print('-',i)
img = tf.keras.utils.array_to_img(generated_images[i])
img.save(f"{current_wdir}/images/{epoch+1:03}_{j}.jpg")
# Supress TF warning messages.
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Check to make sure GPU will be used.
tf.config.list_physical_devices('GPU')
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
gan = GAN(discriminator = discriminator, generator = generator, latent_dim = latent_dim)
gan.compile(
d_optimizer=tf.keras.optimizers.Adam(learning_rate = 0.0001, beta_1 = 0.5, beta_2 = 0.9),
g_optimizer=tf.keras.optimizers.Adam(learning_rate = 0.0002, beta_1 = 0.5, beta_2 = 0.9),
loss_fn = tf.keras.losses.BinaryCrossentropy(from_logits = True))
with tf.device('/device:GPU:0'):
history_gan = gan.fit(monet_transformed,
epochs = epochs,
callbacks = [GANMonitor(num_img = num_img_saved, latent_dim = latent_dim)])
Epoch 1/250
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1727903939.355562 131 service.cc:145] XLA service 0x7d2888002120 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: I0000 00:00:1727903939.355642 131 service.cc:153] StreamExecutor device (0): Tesla P100-PCIE-16GB, Compute Capability 6.0 W0000 00:00:1727903939.713769 131 random_ops.cc:59] Warning: Using tf.random.uniform with XLA compilation will ignore seeds; consider using tf.random.stateless_uniform instead if reproducible behavior is desired. random_uniform/RandomUniform I0000 00:00:1727903965.095241 131 device_compiler.h:188] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process.
19/19 ━━━━━━━━━━━━━━━━━━━━ 61s 2s/step - d_loss: 0.5995 - g_loss: 0.7060 Epoch 2/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 416ms/step - d_loss: 0.5243 - g_loss: 0.7792 Epoch 3/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.4333 - g_loss: 0.8907 Epoch 4/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 416ms/step - d_loss: 0.4048 - g_loss: 1.1171 Epoch 5/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.4802 - g_loss: 1.2327 Epoch 6/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6898 - g_loss: 1.0059 Epoch 7/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6458 - g_loss: 1.2977 Epoch 8/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6970 - g_loss: 0.8350 Epoch 9/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6178 - g_loss: 1.4931 Epoch 10/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6441 - g_loss: 0.8571 Epoch 11/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7234 - g_loss: 1.3230 Epoch 12/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6301 - g_loss: 1.1793 Epoch 13/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6215 - g_loss: 1.1822 Epoch 14/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6844 - g_loss: 1.0288 Epoch 15/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.5528 - g_loss: 1.8988 Epoch 16/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6483 - g_loss: 1.5720 Epoch 17/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6535 - g_loss: 0.8657 Epoch 18/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7057 - g_loss: 0.9742 Epoch 19/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6190 - g_loss: 0.7501 Epoch 20/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6663 - g_loss: 1.0221 Epoch 21/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6832 - g_loss: 1.1714 Epoch 22/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6876 - g_loss: 1.0478 Epoch 23/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6790 - g_loss: 0.9732 Epoch 24/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6563 - g_loss: 1.0403 Epoch 25/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 10s 419ms/step - d_loss: 0.6304 - g_loss: 0.9598 Epoch 26/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6384 - g_loss: 0.8944 Epoch 27/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6799 - g_loss: 0.7423 Epoch 28/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7478 - g_loss: 1.0476 Epoch 29/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6262 - g_loss: 0.7712 Epoch 30/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7038 - g_loss: 0.8086 Epoch 31/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7019 - g_loss: 0.8409 Epoch 32/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6774 - g_loss: 0.7618 Epoch 33/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7162 - g_loss: 0.7582 Epoch 34/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6758 - g_loss: 0.7831 Epoch 35/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 424ms/step - d_loss: 0.6756 - g_loss: 0.9262 Epoch 36/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.6616 - g_loss: 0.7506 Epoch 37/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6739 - g_loss: 0.8385 Epoch 38/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6175 - g_loss: 0.9005 Epoch 39/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.5860 - g_loss: 1.0323 Epoch 40/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 9s 418ms/step - d_loss: 0.7796 - g_loss: 0.8628 Epoch 41/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6822 - g_loss: 0.8258 Epoch 42/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7023 - g_loss: 1.0893 Epoch 43/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6789 - g_loss: 1.0260 Epoch 44/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6988 - g_loss: 0.8076 Epoch 45/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6752 - g_loss: 0.9751 Epoch 46/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6707 - g_loss: 0.8030 Epoch 47/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6805 - g_loss: 0.8056 Epoch 48/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7974 - g_loss: 0.8561 Epoch 49/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7009 - g_loss: 0.7527 Epoch 50/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6782 - g_loss: 0.8860 Epoch 51/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6883 - g_loss: 0.8074 Epoch 52/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6801 - g_loss: 0.9355 Epoch 53/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6755 - g_loss: 0.8392 Epoch 54/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6848 - g_loss: 0.8783 Epoch 55/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6637 - g_loss: 0.8493 Epoch 56/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7450 - g_loss: 0.9564 Epoch 57/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6790 - g_loss: 0.9146 Epoch 58/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6717 - g_loss: 0.9516 Epoch 59/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7065 - g_loss: 0.7331 Epoch 60/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 424ms/step - d_loss: 0.6742 - g_loss: 0.8924 Epoch 61/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7088 - g_loss: 0.8081 Epoch 62/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6741 - g_loss: 0.8263 Epoch 63/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6826 - g_loss: 0.8423 Epoch 64/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6739 - g_loss: 0.9185 Epoch 65/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6523 - g_loss: 0.7841 Epoch 66/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6284 - g_loss: 0.8586 Epoch 67/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6374 - g_loss: 1.0529 Epoch 68/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.6700 - g_loss: 1.2820 Epoch 69/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6660 - g_loss: 1.3240 Epoch 70/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6689 - g_loss: 0.9486 Epoch 71/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6877 - g_loss: 0.9281 Epoch 72/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6792 - g_loss: 1.0678 Epoch 73/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7038 - g_loss: 0.9430 Epoch 74/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6771 - g_loss: 0.9297 Epoch 75/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7036 - g_loss: 0.8331 Epoch 76/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7028 - g_loss: 0.8204 Epoch 77/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7086 - g_loss: 0.6908 Epoch 78/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6983 - g_loss: 0.6555 Epoch 79/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6984 - g_loss: 0.7405 Epoch 80/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6732 - g_loss: 0.7492 Epoch 81/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6856 - g_loss: 0.7898 Epoch 82/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7125 - g_loss: 0.8927 Epoch 83/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6991 - g_loss: 0.8302 Epoch 84/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6966 - g_loss: 0.6960 Epoch 85/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6975 - g_loss: 0.7516 Epoch 86/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7027 - g_loss: 0.7847 Epoch 87/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6611 - g_loss: 0.7931 Epoch 88/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6804 - g_loss: 0.9423 Epoch 89/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6798 - g_loss: 0.8386 Epoch 90/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 424ms/step - d_loss: 0.6982 - g_loss: 0.7087 Epoch 91/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6627 - g_loss: 0.8166 Epoch 92/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6764 - g_loss: 0.8547 Epoch 93/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 10s 419ms/step - d_loss: 0.7288 - g_loss: 0.8773 Epoch 94/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6825 - g_loss: 0.7086 Epoch 95/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6987 - g_loss: 0.7329 Epoch 96/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7006 - g_loss: 0.7386 Epoch 97/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7035 - g_loss: 0.6891 Epoch 98/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6952 - g_loss: 0.7165 Epoch 99/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6893 - g_loss: 0.6849 Epoch 100/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7324 - g_loss: 0.7419 Epoch 101/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6841 - g_loss: 0.7430 Epoch 102/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6805 - g_loss: 0.7382 Epoch 103/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6843 - g_loss: 0.7030 Epoch 104/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6991 - g_loss: 0.7525 Epoch 105/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.7035 - g_loss: 0.7139 Epoch 106/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7056 - g_loss: 0.6788 Epoch 107/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6975 - g_loss: 0.7715 Epoch 108/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7022 - g_loss: 0.6830 Epoch 109/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6860 - g_loss: 0.6966 Epoch 110/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7075 - g_loss: 0.7270 Epoch 111/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6850 - g_loss: 0.7324 Epoch 112/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6967 - g_loss: 0.7485 Epoch 113/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6960 - g_loss: 0.7539 Epoch 114/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6879 - g_loss: 0.7514 Epoch 115/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7537 - g_loss: 0.7860 Epoch 116/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7048 - g_loss: 0.7390 Epoch 117/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6991 - g_loss: 0.7073 Epoch 118/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6911 - g_loss: 0.8152 Epoch 119/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.7017 - g_loss: 0.7504 Epoch 120/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6945 - g_loss: 0.7196 Epoch 121/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7167 - g_loss: 0.7312 Epoch 122/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6925 - g_loss: 0.8971 Epoch 123/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6895 - g_loss: 0.7018 Epoch 124/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6973 - g_loss: 0.7625 Epoch 125/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6950 - g_loss: 0.7437 Epoch 126/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6999 - g_loss: 0.7095 Epoch 127/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 10s 418ms/step - d_loss: 0.6945 - g_loss: 0.6996 Epoch 128/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7030 - g_loss: 0.7239 Epoch 129/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7057 - g_loss: 0.7000 Epoch 130/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6938 - g_loss: 0.6964 Epoch 131/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7931 - g_loss: 0.9981 Epoch 132/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6914 - g_loss: 0.7656 Epoch 133/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7142 - g_loss: 0.7120 Epoch 134/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6901 - g_loss: 0.6991 Epoch 135/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6903 - g_loss: 0.7597 Epoch 136/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7850 - g_loss: 0.8632 Epoch 137/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7037 - g_loss: 0.7739 Epoch 138/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 9s 420ms/step - d_loss: 0.7015 - g_loss: 0.8160 Epoch 139/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7089 - g_loss: 0.7047 Epoch 140/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6990 - g_loss: 0.6592 Epoch 141/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 424ms/step - d_loss: 0.6988 - g_loss: 0.6880 Epoch 142/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7082 - g_loss: 0.7167 Epoch 143/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6958 - g_loss: 0.6934 Epoch 144/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6856 - g_loss: 0.7042 Epoch 145/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6943 - g_loss: 0.6585 Epoch 146/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6943 - g_loss: 0.6892 Epoch 147/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6925 - g_loss: 0.6807 Epoch 148/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6928 - g_loss: 0.6965 Epoch 149/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6890 - g_loss: 0.7263 Epoch 150/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7119 - g_loss: 0.7282 Epoch 151/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.6665 - g_loss: 1.0832 Epoch 152/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7012 - g_loss: 0.7956 Epoch 153/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6918 - g_loss: 0.7261 Epoch 154/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6978 - g_loss: 0.7098 Epoch 155/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7008 - g_loss: 0.9592 Epoch 156/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6870 - g_loss: 0.7014 Epoch 157/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.6915 - g_loss: 0.7787 Epoch 158/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.7014 - g_loss: 0.6863 Epoch 159/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6947 - g_loss: 0.7183 Epoch 160/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7008 - g_loss: 0.6981 Epoch 161/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6943 - g_loss: 0.6933 Epoch 162/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6998 - g_loss: 0.7048 Epoch 163/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6934 - g_loss: 0.6849 Epoch 164/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7023 - g_loss: 0.7390 Epoch 165/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6957 - g_loss: 0.7058 Epoch 166/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6918 - g_loss: 0.7010 Epoch 167/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7164 - g_loss: 0.7485 Epoch 168/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6989 - g_loss: 0.7023 Epoch 169/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6918 - g_loss: 0.7291 Epoch 170/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7009 - g_loss: 0.7789 Epoch 171/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7038 - g_loss: 0.7277 Epoch 172/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6825 - g_loss: 0.6790 Epoch 173/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6924 - g_loss: 0.7991 Epoch 174/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7046 - g_loss: 0.7772 Epoch 175/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6946 - g_loss: 0.7109 Epoch 176/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6860 - g_loss: 0.7293 Epoch 177/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7023 - g_loss: 0.8418 Epoch 178/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6849 - g_loss: 0.7553 Epoch 179/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6887 - g_loss: 0.8320 Epoch 180/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7162 - g_loss: 0.7077 Epoch 181/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 422ms/step - d_loss: 0.7027 - g_loss: 0.7301 Epoch 182/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7041 - g_loss: 0.7223 Epoch 183/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7202 - g_loss: 0.7429 Epoch 184/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7032 - g_loss: 0.7834 Epoch 185/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6992 - g_loss: 0.6879 Epoch 186/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6963 - g_loss: 0.6755 Epoch 187/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6995 - g_loss: 0.6724 Epoch 188/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 424ms/step - d_loss: 0.6895 - g_loss: 0.7052 Epoch 189/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6989 - g_loss: 0.6796 Epoch 190/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6910 - g_loss: 0.7224 Epoch 191/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6980 - g_loss: 0.6790 Epoch 192/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6928 - g_loss: 0.7007 Epoch 193/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.7003 - g_loss: 0.7596 Epoch 194/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.7027 - g_loss: 0.8983 Epoch 195/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7044 - g_loss: 0.7506 Epoch 196/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6940 - g_loss: 0.6767 Epoch 197/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6963 - g_loss: 0.8181 Epoch 198/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6890 - g_loss: 0.6946 Epoch 199/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6910 - g_loss: 0.6763 Epoch 200/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.7095 - g_loss: 0.7940 Epoch 201/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6859 - g_loss: 0.7048 Epoch 202/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7209 - g_loss: 0.7057 Epoch 203/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6930 - g_loss: 0.7200 Epoch 204/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7118 - g_loss: 0.7212 Epoch 205/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.7359 - g_loss: 0.8513 Epoch 206/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6756 - g_loss: 0.7476 Epoch 207/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.7004 - g_loss: 0.7191 Epoch 208/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7035 - g_loss: 0.7414 Epoch 209/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6919 - g_loss: 0.7030 Epoch 210/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 420ms/step - d_loss: 0.7285 - g_loss: 0.7139 Epoch 211/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6932 - g_loss: 0.7280 Epoch 212/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6923 - g_loss: 0.7073 Epoch 213/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6954 - g_loss: 0.6527 Epoch 214/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7050 - g_loss: 0.6984 Epoch 215/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7004 - g_loss: 0.7032 Epoch 216/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6932 - g_loss: 0.7137 Epoch 217/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.7011 - g_loss: 0.6766 Epoch 218/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 417ms/step - d_loss: 0.6915 - g_loss: 0.7087 Epoch 219/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.7835 - g_loss: 0.8190 Epoch 220/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6840 - g_loss: 0.7050 Epoch 221/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6913 - g_loss: 0.6939 Epoch 222/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6988 - g_loss: 0.6882 Epoch 223/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 423ms/step - d_loss: 0.6953 - g_loss: 0.7271 Epoch 224/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 10s 419ms/step - d_loss: 0.6903 - g_loss: 0.6656 Epoch 225/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6964 - g_loss: 0.6751 Epoch 226/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 419ms/step - d_loss: 0.6952 - g_loss: 0.6585 Epoch 227/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6986 - g_loss: 0.6432 Epoch 228/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6977 - g_loss: 0.6799 Epoch 229/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.6902 - g_loss: 0.7503 Epoch 230/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 8s 418ms/step - d_loss: 0.7005 - g_loss: 0.7136 Epoch 231/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 15s 808ms/step - d_loss: 0.7005 - g_loss: 0.7074 Epoch 232/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 557ms/step - d_loss: 0.6989 - g_loss: 0.6910 Epoch 233/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6937 - g_loss: 0.6667 Epoch 234/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 559ms/step - d_loss: 0.7009 - g_loss: 0.6801 Epoch 235/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 557ms/step - d_loss: 0.7071 - g_loss: 0.7210 Epoch 236/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6984 - g_loss: 0.6520 Epoch 237/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 559ms/step - d_loss: 0.7038 - g_loss: 0.6639 Epoch 238/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 557ms/step - d_loss: 0.6948 - g_loss: 0.7839 Epoch 239/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6966 - g_loss: 0.7014 Epoch 240/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6804 - g_loss: 0.7229 Epoch 241/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 559ms/step - d_loss: 0.7481 - g_loss: 0.7537 Epoch 242/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 559ms/step - d_loss: 0.7272 - g_loss: 0.7584 Epoch 243/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 559ms/step - d_loss: 0.7036 - g_loss: 0.6919 Epoch 244/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 562ms/step - d_loss: 0.7078 - g_loss: 0.7371 Epoch 245/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6785 - g_loss: 0.7327 Epoch 246/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.6924 - g_loss: 0.7303 Epoch 247/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 558ms/step - d_loss: 0.7446 - g_loss: 0.8061 Epoch 248/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 560ms/step - d_loss: 0.7063 - g_loss: 0.7100 Epoch 249/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 557ms/step - d_loss: 0.6967 - g_loss: 0.7071 Epoch 250/250 19/19 ━━━━━━━━━━━━━━━━━━━━ 11s 557ms/step - d_loss: 0.6997 - g_loss: 0.6750
Now that the model has finished training over the course of 250 epochs, let's plot the loss values for both the Generator and Discriminator throughout training.
history_gan_df = pd.DataFrame({'D_Loss': history_gan.history['d_loss'], 'G_Loss': history_gan.history['g_loss']})
sns.lineplot(history_gan_df)
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.title('GAN Training History')
plt.axhline(y = 0.7, c = 'k', alpha = 0.4)
plt.savefig('./Generated_Images/GAN_History.jpg')
plt.show()
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
/opt/conda/lib/python3.10/site-packages/seaborn/_oldcore.py:1119: FutureWarning: use_inf_as_na option is deprecated and will be removed in a future version. Convert inf values to NaN before operating instead.
with pd.option_context('mode.use_inf_as_na', True):
A few things to note:
It's also important to note that while the loss stabilized, it still jitters which isn't necessarily a bad thing. This means each model is potentially still learning. However, again, this process is very subjective and close analysis is the only thing that can be done to determine when to stop training the model. 250 epochs was chosen after much trial an error, analyzing the generated images at each epoch.
Speaking of which, now let's plot a generated image from every 5 epochs.
# Load generated images from each epoch.
gen_images_files = os.listdir(f"{current_wdir}/Generated_Images")
gen_images = {}
for file in gen_images_files:
# Skip non-image files.
if '.jpg' not in file:
continue
# Separate epoch number, image number, and file extension.
epoch, _ = file.split('_')
if '0' not in _:
continue
img = Image.open(f"{current_wdir}/Generated_Images/{file}")
gen_images[epoch] = np.asarray(img)
# Plot an image from every 5 epochs.
fig, ax = plt.subplots(int((len(range(5, epochs+1, 5)) / 5)), 5, figsize = (10,int(epochs/10)), sharex = True, sharey = True)
for i, idx in enumerate(range(5, epochs+1, 5)):
ax[i // 5, i % 5].imshow(gen_images[f'{str(idx).zfill(3)}'])
ax[i // 5, i % 5].set_title(f'{idx}')
plt.savefig('./Generated_Images/5_Epoch_Images.jpg')
plt.show()
Now, let's take a closer look at the last 20 epochs to get a better idea of the generative abilities of this model.
# Plot a generated image from the last 20 epochs.
last_gen_img_idx = range(epochs-19, epochs+1, 1)
fig, ax = plt.subplots(4, 5, sharex = True, sharey = True)
for i, idx in enumerate(last_gen_img_idx):
ax[i // 5, i % 5].imshow(gen_images[f'{str(idx).zfill(3)}'])
ax[i // 5, i % 5].set_title(f'{idx}')
ax[i // 5, i % 5].axis('off')
#fig.tight_layout()
plt.savefig('./Generated_Images/Last_20_Images.jpg')
plt.show()
These look great! When looking through the full Monet dataset you definitely can see where these generated images are taking qualities from.
Finally, let's talk about this model's "skeleton in the closet" so to speak...
It seems to suffer from Mode Collapse.
To demonstrate this we will randomly generate 6 latent vectors and input them through the Generator to generate 6 images from the trained and finished model.
fig, ax = plt.subplots(2, 3, sharex = True, sharey = True)
prediction = generator.predict(np.random.normal(size = (6,latent_dim)))
prediction = ((prediction + 1) * 127.5).astype(np.uint8)
for i in range(6):
ax[i // 3, i % 3].imshow(prediction[i])
plt.show()
1/1 ━━━━━━━━━━━━━━━━━━━━ 2s 2s/step
It's immediately apparent that each of these images are in the same "style" with the same color pallette. At each epoch the model would switch "styles" and colors but only every produce one type during that epoch. This is typically indicative of Mode Collapse which usually shows itself in this way -- where the Generator produces a very limited set of outputs. Many things can cause Mode Collapse and is what makes training GANs so difficult as it can be quite prevalent.
Causes of Mode Collapse (in no particular order):
More information on this subject and sources for some of this can be found in the Appendix at the end.
I built countless alterations to this final model to address some of these issues and I believe I narrowed it down quite well, but did not overcome this issue (more on what was done for the final submission for the competition at the end of this section). Below are a summary and general path I took to attempt to solve this issue.
Methods Attempted to Mitigate Mode Collapse:
GAN class, train_step function.bias = false. Noticed many CycleGANs utilize this, but never found a source for the reason.There is quite a lot more information that I haven't touched on this subject, but let's leave it here for now and complete the project.
Now to prepare the submission.
# generated_images = generator.predict(np.random.normal(size = (7000,latent_dim)))
# generated_images = (generated_images + 1) * 127.5
# for i in range(7000):
# img = tf.keras.utils.array_to_img(generated_images[i])
# img.save(f"{current_wdir}/images/{epochs}_{i}.jpg")
import shutil
shutil.make_archive("/kaggle/working/images", 'zip', "/kaggle/working/images")
'/kaggle/working/images.zip'
As mentioned above, even though the final model still suffered from mode collapse, upon analyzing the generated images during training, I decided there was still a way to submit a variety of images.
Each epoch saw the Generator produce images with a new style (mode), and since training of the final epochs saw minimal differences between these final epochs -- Images were generated and sampled the final 20 epochs.
Here is a sample of those images.
last_20_files = os.listdir(f"{current_wdir}/images")
rand_last_20_images = []
for file in np.random.choice(last_20_files, 100, replace = False):
# Skip non-image files.
if '.jpg' not in file:
continue
# Separate epoch number, image number, and file extension.
epoch, num = file.split('_')
img = Image.open(f"{current_wdir}/images/{file}")
rand_last_20_images.append(np.asarray(img))
fig, ax = plt.subplots(20, 5, figsize = (10,40), sharex = True, sharey = True)
for i, img in enumerate(rand_last_20_images):
ax[i // 5, i % 5].imshow(img)
plt.show()
And as a reminder of a few of the Monet paintings for comparison...
for img in monet:
fig, ax = plt.subplots(2, 5, figsize = (10,4), sharex = True, sharey = True)
img = ((img.numpy() + 1) * 127.5).astype(np.uint8)
for i in range(10):
ax[i // 5, i % 5].imshow(img[i])
break
Is it going to fool any art buffs and sell for millions of dollars? - No. However, besides the remaining artifcating from the convolutions that makes it look like monet suddenly got into quilting, the generated images do look distinctly Monet-esque. In my opinion they capture the color blocking and combinations of his paintings and even some of his textures and way he layers paint.
While this seems like a bit of a work-around, all previously submitted models were only from generated images from the last epoch. You can see below that the previous version scored ~18 higher (Version 12, same architecture only difference is images submitted and randomness, of course). Also, we can see a few previous versions of this model before making the adjustments listed above.
Below you can find a screenshot of all the results from the Kaggle submissions.

Alongside MiFID scores let's look at a sample of a generated image from each of these model versions.
Ultimately, the performance of the resultant DCGAN through the iterative process described within this project improved to a respectable level. Receiving a score of ~128.65 and placing 53rd is a good result in this case. Through further tweaks the mode collapse that this model suffered from can be overcome and increase performance further.
Of course, a model that produces a true style-transfer like pix2pix/CycleGAN, training a Generator that uses an auto-encoder or UNET architecture that is able to receive an image as an input and generate from that image, is going to score better than a DCGAN in this competition, but I found this exercise of attempting randomly learned generation to be a valuable one.
More specifically, the difficulties in training a GAN can come from all directions. Including:
GANs can be difficult to build and tune properly. The subjective nature of building and training a DCGAN, or any GAN for that matter, that generates Monet-esque images from random noise latent vectors is its own kind of art.
Resources that helped along the way in no particular order.
Identifying unstable GANs and their failure modes https://machinelearningmastery.com/practical-guide-to-gan-failure-modes/
Exported to HTML via command line using:
jupyter nbconvert Adversarial_Painting.ipynb --to html
jupyter nbconvert Adversarial_Painting.ipynb --to html --HTMLExporter.theme=dark